System and method for auto-focusing an image

ABSTRACT

A method and system for auto-focusing is disclosed where the number of the fractal pixels having a fractal dimension within a predetermined range is determined. The focused image is determined by adjusting the image to maximize the fractal pixel count. Alternatively, the focused image may be selected from a set of images, the image having the largest fractal pixel count selected as the focused image.

CROSS REFERENCE

This application is a continuation of U.S. patent application Ser. No.10/368,055, filed on Feb. 14, 2003, now U.S. Pat. No. 6,876,776.

FIELD OF THE INVENTION

The present invention relates to digital imaging. More specifically, theinvention relates to systems and methods for automated focusing of animage.

BACKGROUND OF THE INVENTION

It is well known that manual evaluation of biological samples is bothslow and highly susceptible to error. It is also well known thatautomating the sample evaluation both increases the sample evaluationrate and reduces error.

FIG. 1 is a schematic of an automated scanning optical microscopysystem. The automated scanning optical microscopy system 100 includes anoptical microscope modified to automatically capture and save images ofa sample 105 placed on a sample holder 107 such as, for example, aslide, which in turn is supported by a stage 110. The optical componentsinclude an illumination source 120, objective lens 124, and camera 128.Housing 130 supports the optical components. The design and selection ofthe optical components and housing are known to one of skill in theoptical art and do not require further description.

The automated system 100 includes a controller that enables the stage110 supporting the slide 107 to place a portion of the sample 105 in thefocal plane of the objective lens. The camera 128 captures an image ofthe sample and sends the image signal to an image processor for furtherprocessing and/or storage. In the example, shown in FIG. 1, the imageprocessor and controller are both housed in a single PC 104 althoughother variations may be used. The mechanical design of the stage 110 isknown to one of skill in the mechanical arts and does not requirefurther description.

The controller may also control a sample handling subsystem 160 thatautomatically transfers a slide 109 between the stage 110 and a storageunit 162.

The controller must also be capable of positioning the sample such thatthe image produced by the camera 128 is in focus. In addition, thecontroller must be able to position the sample very rapidly in order toreduce the time required to capture an image.

One method of auto-focusing an image employs a laser range finder. Thecontroller calculates the distance to a surface based on the signal fromthe laser range finder. The advantage of such a system is that it isvery fast, thereby increasing the scan rate of the automated system. Thedisadvantage of such a system is that it requires additional hardwarethat may interfere with the optical performance of the automated system.A second disadvantage of such a system is the inability to focusdirectly on the feature of interest in the sample. The signal from thelaser range finder is usually based on the highest reflective surfaceencountered by the laser beam. This surface is usually the cover slip orthe slide and not the sample.

Another method of auto-focusing an image employs image processing todetermine when the image is in focus or, alternatively, select the mostfocused image from a set of images taken at different sample-objectivelens distances. The advantage of using image processing to auto-focus isthat it can focus directly on the sample instead of the slide or coverslip. The disadvantage of image processing auto-focus is that it usuallyrequires large computational resources that may. limit the scan rate ofthe automated system. The large computational requirement arises becauseprior art algorithms based on maximizing the high frequency powerspectrum of the image or on detecting and maximizing edges must performlarge numbers of computations.

Therefore, there remains a need for a rapid auto-focusing method thatdoes not require large computational resources and can directly focusthe sample.

SUMMARY OF THE INVENTION

One embodiment of the present invention provides a method forautomatically focusing an object comprising: acquiring at least oneimage of the object, the image characterized by a focal distance andcomprising at least one pixel; determining a fractal pixel count ofpixels having a fractal dimension within a predetermined range; andselecting an optimal focal distance from the image having the largestfractal pixel count.

Another embodiment of the present invention provides an apparatus forautomatically focusing an object comprising: an image capture sensor forcapturing an image of the object, the captured image characterized by afocal distance; a fractal estimator coupled to the image capture sensor,the fractal estimator adapted to estimate a fractal dimension of atleast one pixel of the captured image; and a focus controller coupled tothe fractal estimator, the focus controller adapted to adjust the focaldistance of the object based on maximizing the number of pixels havingan estimated fractal dimension within a predetermined range.

Another embodiment of the present invention provides an apparatus forautomatically focusing an object comprising: an image capture sensor forcapturing an image of the object, the captured image characterized by afocal distance; means for determining a fractal pixel count of thecaptured image; and means for controlling the focal distance based uponmaximizing the fractal pixel count of the captured image.

Another embodiment of the present invention is directed to a computerprogram product for use with an automated microscopy system, thecomputer program product comprising: a computer usable medium havingcomputer readable program code means embodied in the computer usablemedium for causing the automated microscopy system to automaticallyfocus an object, the computer program product having: computer readablecode means for causing a computer to acquire an image of the object at afocal distance; computer readable code means for causing the computer todetermine a fractal pixel count of the acquired image; and computerreadable code means for causing the computer to adjust the focaldistance of the object based upon maximizing the fractal pixel count ofthe acquired image.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described by reference to the preferred andalternative embodiments thereof in conjunction with the drawings inwhich:

FIG. 1 is a schematic diagram of a prior art automated scanning opticalmicroscopy system;

FIG. 2 is a flowchart of an embodiment of the present invention;

FIG. 3 is a flowchart illustrating the generation of a boundary image inan embodiment of the present invention;

FIG. 4 a is a diagram illustrating an L=3 structuring element used inone embodiment of the present invention;

FIG. 4 b is a diagram illustrating an L=3 structuring element used inanother embodiment of the present invention;

FIG. 5 is a graph of C_(D) vs. S produced by the embodiment shown inFIG. 2.

DETAILED DESCRIPTION OF THE PREFERRED AND ALTERNATIVE EMBODIMENTS

An underlying idea supporting current auto-focusing algorithms is thatan image in focus will exhibit the largest pixel-to-pixel variations(the difference of pixel values between adjacent pixels) whereas anout-of-focus image will blur, or reduce, the pixel-to-pixel variationsin the image. Autocorrelation or power spectrum algorithms are designedto measure and/or maximize the high frequency (variations occurring overa small number of pixels) component of an image. Autocorrelation,however, is a computationally intensive process that is prohibitive whenconsidered for high scanning rate automated optical microscopy.

Maximum gradient methods avoid performing an autocorrelation bydetecting edges and maximizing the variation across the edge. Themaximum gradient method calculates the differences between adjacentpixels along a pre-selected direction and several directions may be usedto find and maximize the pixel gradient across edges. The ability of themaximum gradient method to find the correct focus decreases as thecontrast and/or brightness of the image decreases because decreasingcontrast or brightness also decreases the differences between pixels.

The inventor has surprisingly discovered a method for auto-focusing thatrequires neither a direct calculation of the pixel-to-pixel variationsof the maximum gradient method nor the computationally intensiveautocorrelation of the power spectrum methods. Instead, the presentinvention analyzes each image at different scale lengths to calculate afractal dimension for each pixel in the image. Furthermore, the fractaldimension is estimated using the pixel values of the neighborhood pixelsinstead of a count of the neighborhood pixels. The fractal dimension isthen used to determine if the image is a focused image.

FIG. 2 is a flow diagram of one embodiment of the present invention. Animage, I₀, is read in step 205. The image may be read directly from acamera or may be read from memory. In one embodiment, the controllercaptures several images of the sample at various sample-objective lensdistances and stores each captured image in an appropriate memory devicesuch as a hard drive, an optical drive, or semiconductor memory. In analternative embodiment, after each image is captured, the process shownin FIG. 2 is completed and only images meeting criteria described beloware saved to a memory device.

In step 210, a first boundary image, I_(B1), is generated from 10 andstored. A second boundary image, I_(B2), is generated from I₀ and storedin step 215.

FIG. 3 is a flow diagram illustrating the generation of each boundaryimage, I_(B). An erosion image, E_(L), is generated from the capturedimage, I⁰, and stored in step 310. A dilation image, D^(L), is generatedfrom 10 and stored in step 320. In step 330, the boundary image, I_(B),is generated by subtracting the erosion image from the dilation image,i.e., I_(B)=D^(L)−E^(L). The superscript, L, in E^(L) and D^(L) bothrefer to the size, or scale, of the structuring element used to performthe erosion or dilation, respectively.

The structuring element may be represented by an L×L matrix comprised ofones and zeros. The structuring element is characterized by an originpixel and a neighborhood. The neighborhood comprises all the matrixelements that are set to one and is contained within the L×L matrix. Animage is generated by calculating a pixel value for the pixel at theorigin of the structuring element based on the pixel values of thepixels in the neighborhood of the structuring element. In the case oferosion, the pixel value of the origin pixel is set to the minimum ofthe pixel values in the neighborhood. Dilation, in contrast, sets thepixel value to the maximum of the pixel values in the neighborhood. Inone embodiment, the neighborhood is coextensive with the structuringelement where the L×L matrix comprised. of all ones, as shown in FIG. 4a for an L=3 structuring element. In another embodiment, theneighborhood is less than the structuring element in that the L×L matrixincludes at least one zero. In another embodiment, the neighborhood is a“cross” or “plus” centered on the origin of the structuring element, asshown in FIG. 4 b for an L=3 structuring element.

Referring to FIG. 2, step 215 generates a second boundary image, I_(B2),at a different scale, L₂, using a different size structuring elementthan the structuring element used to generate I_(B1). The selection ofthe scale for both boundary images may depend on the feature size of theobject of interest, the computational limitations of the processor, andother such factors as is apparent to one of skill in the art. In oneembodiment, L₁ and L₂ are selected to maximize the difference between L₁and L₂ under constraints such as those identified above. In oneembodiment, L₁ may be chosen from the group consisting of greater than2, 3, 4, 5, and-greater than 5. In a preferred embodiment, the scale ofI_(B1) is set to L₁=3 because L=3 represents the minimum, non-trivialbalanced (in the sense that the origin is centered in .the structuringelement) structuring element. L₂ is selected such that L₂ is greaterthan L₁, or, stated differently, the ratio, R=L₂/L₁>1. In oneembodiment, R is in the range selected from a group consisting of 1-4,4-8, 8-16, and greater than 16. In a preferred embodiment, R=7.

The fractal dimension, d_(p), for each pixel in I₀ is estimated from theboundary images I_(B1) and I_(B2) in step 220. The fractal dimension foreach pixel may be estimated by the equation (1):

$\begin{matrix}{d_{p} = \frac{\log\left( \frac{N_{2}}{N_{1}} \right)}{\log\left( \frac{L_{2}}{L_{1}} \right)}} & (1)\end{matrix}$where N₂ represents the sum of the pixel values in the neighborhood ofthe structuring element centered on the pixel in I_(B2) and N₁represents the sum of the pixel values in the neighborhood of thestructuring. element centered on the pixel in I_(B1).

In an alternative embodiment, the fractal map is generated directly fromI₀. The fractal dimension for a pixel is estimated by centering an L₁×L₁ structuring element on the pixel and summing the pixel values of thepixels within the structuring element to form a first sum, N₁. A secondstructuring element of size L₂×L₂, where L₂>L₁, is centered. on thepixel and a second sum, N₂, of the pixel values of the pixels within thesecond structuring element is calculated. The fractal dimension of thepixel is estimated using the equation

$\begin{matrix}{d_{p} = \frac{\log\left( {N_{2}/N_{1}} \right)}{\log\left( {L_{2}/L_{1}} \right)}} & (2)\end{matrix}$where d_(p) is the fractal dimension of the pixel in I₀, N₂ is the sumof the pixel values in the L₂×L₂ structuring element, N₁ is the sum ofthe pixel values in the L₁×L₁ structuring element, and L₂ and L₁, arethe sizes (in pixels) of the respective structuring elements.

The form of equation (1) clearly shows that the fractal dimension isestimated by taking ratios of pixel values and therefore should providea more robust method than maximum gradient methods of identifyingout-of-focus images in low light or low contrast conditions.Furthermore, it is believed that the use of sums in N₁ and N₂ reducesthe statistical variations that may be expected in low light conditions.

The inventor has discovered that images containing many pixels having afractal dimension within a range of values tend to be more in-focus thanimages containing few pixels having fractal dimensions within the range.

The predetermined range of fractal dimensions may be determined by oneof skill in the art by visually examining a group of images andselecting the range such that at least one image but not all the imagesare considered to be nearly in-focus. Setting the range narrowly mayexclude all the images whereas setting the range broadly reduces theeffectiveness of this test. In one embodiment, the predetermined rangeis between 1 and 2. In a preferred embodiment, the predetermined rangeis chosen from a group consisting of 1-1.25, 1.25-1.5, 1.5-1.75, 1.75-2,1.5-2, and 1.6-2.

In one embodiment, the predetermined range is determined by an operatorbefore auto-focusing operations. In an alternate embodiment, thepredetermined range may be set to a default range and dynamicallyadjusted during auto-focusing operations to produce at least onein-focus image.

After d_(p) is estimated for each pixel in I₀, a count of the fractalpixels in the image is performed in step 230. A fractal pixel is a pixelhaving a fractal dimension within the predetermined range. The fractalpixel count, C_(d), is the number of fractal pixels in the image.

After C_(d) is determined for the captured image, step 235 checks foradditional images. If there are additional images, control jumps back tostep 205 to repeat the process for the additional image. If there are nomore additional images, the focus image is determined in step 240 byselecting the image having the largest C_(d) from the set of images.

FIG. 5 is a graph illustrating the selection of the in-focus image froma set of images taken at different focal distances. In the example shownin FIG. 5, 9 slices, or images taken at progressively larger focaldistances are numbered from 1 to 9 on the horizontal axis. The fractalpixel count for each slice is shown on the ordinate axis. In thisexample, slice 5 contains the highest number of fractal. pixels of the 9slices and is selected as the in-focus image. The slice 5 image 501 isshown in FIG. 5, along with the slice 3 image 505, which is anout-of-focus image. A fractal dimension map 502 shows the estimatedfractal dimension for each pixel in the slice 5 image. Fractal pixelimages 503 506 show the fractal pixels in the slice 5 and slice 3images, respectively.

Having described at least illustrative embodiments of the invention,various modifications and improvements will readily occur to thoseskilled in the art and are intended to be within the scope of theinvention. Accordingly, the foregoing description is by way of exampleonly and is not intended as limiting. The invention is limited only asdefined in the following claims and the equivalents thereto.

1. An automated system for imaging of biological samples, comprising: aslide storage unit; a microscope stage; a sample handling subsystem forautomatically transferring a slide from said slide storage unit to saidmicroscope stage; an optical microscope coupled to an image capturesensor to capture images of an object; a computer containing a computerprogram embodied in a computer readable medium for automaticallyselecting an in-focus image of an object, comprising: obtaining a set ofimage slices of an object at different focal distances; determining afractal pixel count of each image slice; and selecting the image sliceof the object displaying the highest pixel count as the in-focus imageof the object.
 2. The automated system of claim 1, wherein the focaldistances of the image slices of an object are obtained incrementally.3. The automated system of claim 1 wherein determining the fractal pixelcount comprises estimating a fractal dimension for at least one pixel ofthe image slice of the object, the fractal dimension of the pixel givenbyd _(p)=log(N ₂ /N ₁)/log(L ₂ /L ₁) where d_(p), is the fractal dimensionof the at least one pixel of the image slice of the object, N₂ is thesum of the pixel values in an L₂×L₂ structuring element, N₁ is the sumof the pixel values in an L₁×L₁ structuring element and L₂ and L₁ arethe sizes (in pixels) of the respective structuring elements.
 4. theautomated system of claim 1 wherein determining further comprises:forming a plurality of boundary images from the acquired image, each ofthe plurality of boundary images characterized by a scale; estimatingthe fractal dimension of at least one pixel of the acquired image fromthe plurality of boundary images; and incrementing the fractal pixelcount if the estimated fractal dimension of the at least one pixel iswithin the predetermined range.
 5. The automated system of claim 4wherein the plurality of boundary images includes at least two boundaryimages.
 6. The automated system of claim 4 wherein determining furthercomprises: forming a plurality of boundary image slices from theobtained image of an object, each of the plurality of boundary imagesslices characterized by scale; estimating the fractal dimension of atleast one pixel of the obtained image slice from the plurality ofboundary images slices of the object; and incrementing the fractal pixelcount if the estimated fractal dimension of the at least one pixel iswithin a predetermined range.
 7. The automated system of claim 1,wherein forming the boundary image slice of an object further comprises:eroding the acquired image by a L×L structuring element to form aneroded image; dilating the acquired image by an L×L structuring elementto form a dilated image; and forming the boundary image by subtractingthe eroded image from the dilated image, the scale of the boundary imagedefined by L.
 8. The automated system of claim 5 wherein the scale of atleast one of the plurality of boundary images is selected from a groupconsisting of 2, 3, 4, 5, and greater than
 5. 9. The automated system ofclaim 7 wherein a second scale is greater than a first scale.
 10. Theautomated system of claim 9 wherein the second scale is determined suchthat a ratio of the second scale to the first scale is in a rangeselected from a group consisting of 1-4, 4-8, 8-16, and greater than 16.11. The automated system of claim 10 wherein the ratio of the secondscale to the first scale is
 7. 12. The automated system of claim 6wherein the structuring element has a neighborhood coextensive with thestructuring element.
 13. The automated system of claim 6 wherein thestructuring element has a neighborhood less than the structuringelement.